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• The following dhapters^describe various aspects of lesson production 

and use of the PEATO system at Chanute AfB. The first chapter ('^Aspects 

/ . ' ^ ' ^ 

of Lesson 'Production at Chanute**) describes those factors that were 

• * 

most influential in affecting the creative process. The second ^("Computer 
Aided Management ajt *6hanute")^concems the use of the PLATO system for ^ ' 



dat4 management ,i^tudent testing, and other related CMI aspects. Thfe 

V 

last chapter (^A Survey of Chanute Lesson Development Techniques") 

discusses the development of eight lessons in specific terms and ho\^ 

these J.es^ons were modified to increase their chances of validation. 
, f 

It is hoped that taken together tl?ese. studies will provide ^ome insight- 
into the. development and utilization ^of CBE materials at one military 
installation. 



/ • 
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Chapter 1: Aspects of Lesson Production at Chanute 

■ I 

Abstract 



The purpose of this study is to examine the process of lesson 
production at^ Chanute AFB durihg >the ^period of ISD involvement. The ^ 

iport considers four major factors influencing lesson production: 
implementation of the "lean approach'*, the ISD role in ^esson^ produc- 



tion, the transfer of programmed instruction techniques to CBE, and the - 

•general method of les«on production (group .vs. individual). Each, 

factor is discussed in terms, of its effect on the quantity and quality 

^ of lesson production. This report was submitted to the Chanute PLATO . 

\ 

author staff for their comments and suggestions. 
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Introduction 



Background and Objectives , 

I • ' 4 - 

This study is intended La describe some aspects of J^sson develop- 
ment . at Chanute. It focuses on the process" as it unfolded dur:^ng an 
imp/o'rtant period of the project's existence: June 1974 to June 1975*. 
This was .a crucial phase 'of the project when the * Instructional System 
Development (ISD) group as sume d^ dd ruction of the PLATO effort.^ ,A ^ 
flfef initive description of the site hi^s^ory is not warranted here. 
|[For further details, consult Himwich, iW 7^ d> Dolman et al . , 19i?) . 

* • ' • \ 

(Only the highlights of the process will be covered in this study. 



It is hoped that the results of this treatment will be generalizfeble to 
' Other PLATO and/6r CBE inst^allat ions . j ^ • • 

Method of Approach ^ . * 

, (^ven the constraints as indicated above, the following factors* will 
f be discussed:- ^ * 

1. implementa'tibn^of the ''lean approach" tj) lesson product4.on; 

I ' . ' , / 

2. ISD involvement in lesson production;* 

3,. tran^sfer of programmed Instruction i^iechniques to* CBE; 

4.^ general method t>f lesson production. 
Specific characteristics of ipdi\y.dual l«?soi|p^ have already been discussed 
(Klecka, 197^7a, 1977b). In the" present study, th6 factors that wei'e 
considered to have the moPBt^^ignif leant effect oh ies^p'prddffction 
at Chanute ware chosen for detailed investigation. ^ 

I 

y / 

/ 



ImplemeotHaftrloa (>f *fhe-'"LV^n Approach" to Lesson Production 



^ Th€ lessotls? ^Wl^^^^^ '(3]h^ifiu&e wer^e, Tptoduced accord 



ins to t4iG 



.ERIC 



basic philosophy of the so,-called "lean approach". This consists of 

, i^ncorporating the 5afe essentials to cover the, training obj ectlve^. vithout> 

any extra "fat" that mi'feht be construed ^as overteaching the studeat. The 

emphasis is on training personnel. Consequently, only enough : text is 

given^*to describe ^the^^ubjefet matter and only a few q-uestions ar6 ask^d -* 

of the student* in. the course of the Idsson. / ^ 

* ^' In Chanute's-lessons, tl^^judeaC^' s progress is me^sujred by a, matching, 

t^s^at^ the end of the lesson:, this is fcrtown as 'the Master Validation' 4 

Exam (MVE) which was used originally for '.insuring that 90%^qf the students 

gc^ng through the lesson scored 80% or better on the test. The test and 

the philosopJjy of the lean approach make a clear distinction between wfaat 

a student NEEDS to know and what is NICE to know but is not^ ESSENTIAL to • 

the performajtice of the task beir^ taught. The .lean aptJroac^h, when 

' 'J - * * « 

strictly aprilied, comprises naed:-frtr-know material onl^ > Cl^he 'emphasis i^ 

necessary s/tnce relerences to the leaa approach. are used justify the 



specific aipects of individual lessons: see Klecka,. 1977a'/* 1977b. T 

The Mssion of the ^Air ^'Ace tethiiical - training is'^to produce at ' ' 
niinimum lost qualified individuals can effectively petform, their duty* 
Thus; tttfe students are gdven ins^tj^ct ion .only for t^ai^r particular field 
of* speCializatioR. Everything within ^is ti;;aining fraftework relates to* 
that Specialized area, at? leasi in theory. There is no interest in giving 
a br6id range of^heory and practice in several" dif ferent a^eas, as Is the 
cas^ in the private sector, such as a' university ^ This would be* . 
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prohibijtively expensive and would serve n6 us^eful purpose, ifl the Air 
Force' s*' view. The airman is prepared^'to^ do a specific job that* and no 
otl\er. ^ , * , . 

C haracteris.tic s ' • " * ! * • * . 

'The general'.' character ist'ifcs ot the lean aj^aroach can be distilled 
\ ' ^ * 

as folAws: v ' " . 

\, presentation of, only material Ii%ai* f3r . ^ 

perfomting a specific task; • W 

2. presentation of the minimum amount of such ^essential . 
material to ensure adequate- perfo^nance'JJi#^tL^ 

• 3. testing the student op 'a .few ^mpoctatit tei;m3/ component 6 
"that are sufficient to indicate familiarity^^th the ; 
subj,ect matter (i.e., the test samples the content , 
' • - area and is not'- compr'ehensive) ; , . / 

' » ' ' ' • ' . 

4. OTiting the materials origiMlly in an overly lean , 
•fashion because overteabhiflg^ is relativ.ely more/ 
- idafficult to*detect than unaerteaching; later the 
}r^sons are '^shored 'up" where necessary to achieve f he 
desirfecl mastery. . ' . , 



TRe specif ie' characteristics'" of the lean approach%hen applied to 

CBE Instruct icjn at Chanifte inc^ud^ the preparatiori of materials 'with 

4 

"no' frills" (Dallma^, 1975/ p . '4) : that is, . 

' . - ' ... ' 

- d. * little' branching e'xcept- for fotced review; ^ »- ^ 

^ 2^ multiple ' choice -questions* atid answers; * m ^' ' ' 

^ 3. tittle f^edb-ack for correct or especially ' for dncGft^ct 
answers; ' . ' ' 

^ ^ ^» few elaborate diagrams' t<^ accoinpany \nd elucidate text; * 

^ ^'little stud'ent control 6f movement within , the lesson; 

I ^ .6- 'sizeable instructional steps, and 

. 7. , perceptible reliance on the techniques of programmed 
'instruction for CBE lesson construction. . ' ' . 



-These characteristics are Significant since CBE l-essons are conparatively 
'expen§,iv;^ to produce; .Thus, iilcluding ^'frills" was expensive, and of> » 
^ 4oubtful .value, in the^Air Force's view. Although' this' philosophy was 



sometiipes distasteful to authors and reviewers, it was defended on the 
basis of ^Cost-ef f ectiveness • ,y , 

discussion • • c ' 

The *'no frills'*, approach to lesson production was implement.ed to,.- 
create Efficient '.lessons quickly: that is, lessons that presented the 
material concisely and without elaborate diagrams or lei'igthy disdussions. 

\ ' ' " . c 

ilowever, some of the lessons examined in th^ individual reviews had 

elaborate, slow-tb-appear diagrams.'^ The pire^ence of such schematics 

would sfeem' to deviate from the ISD phil6so{>hy; this is especially true ' 

sioce compt^hension of the diagrams seemed unnecessary for successfully 

complex in^ the fin^ exam. ' 

" The lean approach philosophy carried over to the area of remediation ^ 

for incorrerct'v answers on individual questions, as well as the ehd-of-lesson 

test itself. In tliese cases, no special help units. wexe constructed ^ 

, . but the stoQent was merely forced back through 'material he had already. 

seen in order to hopefully comprehend it the s^ond time around. The 

amount of space required for the lesson to be studied on-li^^ould remain 

the same when this type .of remediation was used. Because implementation 

of this technique reduced or at least did not worsen the coirjputer storage 

^problem, it was a favored approachj More /important, it did not require 

the .creation of new material and was thus* quicker and more efficient than 

* 

constructing additional help units. 
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Examples of a deviation from the lean appro'acti have been derCumentfed 
(Klecka, 1977a) ^ Occas^Lonally , text was presented which wSs not tested 
^ directly and thus did not fall within the lesson* objective of passing * 
the MVE. Conversely, lesson objective, being very broad, implied, that 
a vast amount of material would be covered* In\^ actuality, o.nly the 
brief,est ii^t reduction to the subject was presented; .Thus, adherence to 
this.< philosophy of lessor! construct!l^n was not universal. 
Summary . • w * ^ 

In theopy, the lean approach sdunds very appropriate for implementing 
Air Force technicail training: to gij^e the airman the minimum but adequate, 
amount * of preparation foF the Actual field experience where he^will realljr 
* learn his, job\ In practi^ce, Chanute authors found it hard to f„ollow that 
approach rigorously and hence som^ aspects of the lesion materi0ls varied 
in emplfasis and adherence to it. Nevertheless, , scattered individual 
reports from other bases (e.g., personal intervi^) have indicated that 
the current level of CBE training is probably adequate for b^sic prepava- 
tion (ft the 'airman. Field evaluations indicate average or above average 
results (Dallman et al . , 1977). However, it does not follow that this 
was due either to jigorous or relaxed applications of the widely-publicized 
lean approach. -♦-^ 
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ISD Role in Lesson Prod 



'reduction ^ 



[nltf^l Involvgmi|pt ' ^ - ' , ^ 

At Chanute during the period cbvered-hy this study (mid '74 to 

mid *75), the Instgrucfclonal System Development (ISD) group direcbed the 
* * 

process by'which the CBE lessons w£?uld be prepared. This group had pr'ima^^y 
responsibility fof a devfelopmfeht^l process 'known as the J'f ront-end 



analysis-'. Tt consisted of ^he .following elements: 

1. preparation of «a task analysis, a speci^ized prx)cess 
which encompasses describing and analyzing the job 
taska^'in oi:de^ to determir* appropriate training 
(Department of tke Air Force, 30 January 1974); 

• 2. dfeterraination of skill levels for performing the 

^required task(s); ^ . . 

3. preparation oi^xG. lesson objectives; , \ 

4. creation of* cr-iterion tfeg:ts to check the performance 
leyel of the 'students . ' , * . ^ 

Th£ PLATO authors were Hof in^^ved in --the preparation of the analyses # 

-nor in the Composition of the Com^oneftt^ parts . . . 

Initially, the ISD staff beliej/ed ;that~ the PLATO aiithor staff. was 

suf-f iciently qualified and. experienced that a further breakdown jqr 

• /• 
guidelines * for either subj'ecE matter or instructional d'esign wduld not | 

b.e needed'. Ind^d, LSlT-est imatSd' >that material^ -coul^ ibe produced within 




a^'Vange of $&M»al?^hours/contact-hour of^ instruction (eatiy inte*-vije\QiiL^ 

project head) Co 100 hours/cpntact-hour (Dallman, 1974, p. 2) • These, 

' /* ■ ' ' 

estimates were made despite the fact that 'a 'preliminary expendit ure^y^F^ 
445 man-hours/cont^ct^houc- (Green, '1973*, p. 27) was required to produce a 
rough draft and 650 hpur^/t:ontact-hour to. .produce a' finished lesson 
(Main et, al., 1973, p. 38~j). Simil-auly, »it was known that Aberdeen 



.reqUire^d slightl^^ less than 300' hours of effort for each contact hour of 
.materlla^ produced (Dare et al., 1975, p. 85). 

In the preliminary stages of the,,T5roject , the*author staff inay have 

\ . , " . 

p 

been especially critical of the ISP prepared materials for two reason^. 
First , J because the, authors had not taken part in the ^ ftont-end -analysis 
an* because of limited con^unication between the groups (e.g., "separate 
offices), the auttiors proceeded' an the basis of the documentation 
furinisj;ied by the ISD staff without the background ^nd rationale behind * 
those documents, j^econd, the ISD staff expected to achieve a dramatic 
speed-up in lesson production by eliminating some of the ^yles, holbits,, 



and strategies \^hich authors tound interesting and motivat^ag^ but which 



ISD found unnecessary. 
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Th.e lesson*' objectives were a'lso prepared in* advance for the authors. 
These usOally consisted of a standard approach toward the material in the 
lesson (i.e., "without reference, identify basic facts- and state general. 
^ opera^irvg principles of 1^*^* ThiS'go6l had to be satisfied by 



passilig an end-of-le sson test with an 80% or higher score. Th^ striking 
similarity in the format of the^ objectives within different lesson^ lies 
dn th^ir common' original th the ISD group and the people 'i^ charge cj^ th^- 
yel^icle course itself. The authorj^-we^ provided with' only the general \ 
'topic of each lesson. They had few guidelines abgut the breadth and dicpth 

.. • • 

of the coverage desired. „Even the end-of-lesson tests were an imperfect 
guide to proj^aring the lesson; tHey> on'ly sampled the lessc^^Hfctent . 



Subseq^enft ^Revisions 

The authors asserted that the material presented to ^ them by ISD was' 
veryt' incomplete and unsatisfactory. This was especially apparent in t]\e 



less^ objectives and *end-of-3»Qsson test^: ' it was the considered 
opinion of , the authors that these components were, not sufficient as ^ 
.guidelines for the preparation of lesson materiais. In view of these 
complaints, the ISD staff reversed its as^sjimptions and took a ^rathpr 
conservative view of the autjjors' subject matter expertise and instructional. 

' ■ ' * ' . ■ 

•design experience. The following .majcg:- 'revisions were subsequently made 
in the method of operation:^ ' / " - . • 

1. 'the end-of^lesson tests (the master validation- exams ^ 
or MVE's) were rewritten by ISD or in some cases 

rewritten by the authot^ and# appr<rved by JSD or 
vehicle course personnel,** * 

2. teaching p6ints, a more, detailed outline of what the 
lesson contained than presented by the lesson ob'jec- 

* tiye, were furnished by^ IStt to the author ^taff; and 

■ : ; , , ' 

, 3. -subject matter "experts (SME's) were brought .into the 

project , to lend their expertise to preparation of ^ 
V* lesson materials. , 

Thus/ delays cau^fed by 'insufficient input from ISD to -the author staff 

ranged from a f^w,^day^ te *a few weeks. These delays came at the commence- 

ment of lesson development, a crucial period for the author staff (Missel t, 

1975'; p. 6). ' ' . J • . 

The^ISD distillation of- raw material, was not without its benefits 

for th^ authors^ They were presented with some, of the material which they 

could shape and mold according to their specific inclinations. That/ i^to ' 

say., the authors did not have ^o ' start from fectatch'* (always a painful ' 

procedure) but had something to Work with right' away (i.e*, the teaching 



points). .Nevertheless, the authors were more highly constrained in the 
raod/f ied'relatiotiship to the ISD staff and raised objections about certa 
ISb cclbiponents (i.e.,. the MVE's mentioned above). Overall, however , the 



I- ■ . 
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ISp g'roup pravided structure .and organization in tkis area of*na*scent 
• CBE deVelopmept^ |k 



'fhixe-was 'al^o some feedback frdm the.JSD groilp during and after 




the lefeson. writing ^tage. The ISD;.p%r^ai«^ei*"'wouM 

, question here /6r^ther4i '^heir comments reifrf op aa.d' the basic ISD stirate^ 
of prodiiaing |.esfohs:j Iles'sons were to be coTnpose'cJ of- narrat^ive portions 



with a', few qti^stiona 



ihtetspeYsed'-'^^Jmewtja't at r'mdom. ^t the dnd of the 



lesspn the $tudenf wc uld^^ke' a ^Jjes^*- to see^if.hfe nffd'met the objective. 
Thqs, the ISD, group jDf.fered mainly * iTlinor"ajdditiQTis oi" phanges' rather than 



'radical, instruct ioriall strat«egy' mbdif Sections ./ ^ \ 



••V 



Sunnn^f y 



It can bei se^fl ttiaC'tne -Kl) ^group of fe&ed -th^ ,authdrs a. 'starting 

V. * \ ' . " ' ' / ' ' - . * ' ^ ' ' ' . 

point for the composition "of the ^«iAfe^rial4« WhaCt feifterg^d^ typically wa^ 

)se Wtlir^^- |(i.e. , the X)bjectiv6s and^^tfeac^iing ^points") was- , 
Dy tl^i|>TSD ^pftt^so,nnel but- with the, ma^JoEity of its spetific , , -r^- 
- aspects beia£ cieVted, b>vthie aatt\o^^ with input' from Qou*rse staff. . ' 

Either, because oft t imp ^conscraiftts'' b^c^use* of /statements' and suggestions 

. - . _ • V M ^ ' . " > . . 

taf f , \efideaYors* i:eqiiiring''lr\ltiai .time iavestmenf: for poten- 



a lesson^ whose 
det^rinin^d by t 

if cieV 



frtjm the ISD s 

tial long-term ^payoff^ were^ not; investigated. ' 

. The ISD group cqnC^erftiiarte^cl on a. low-risk, law-pdtential-gai^n. method 

. ^ ' . \' ^ V«^'\ . ^ * . ' " , ^ ' 

ofi^ operation.^ .This apprckcB\ was probably jjls t if ied b.y the project objec^, 
tived. ai^d tJhe 'hipjdory of the^ site. A knowledge /of that history aids the' • 

• ■ .1 , - 

* » . , " ' 

understanding of wKy Ifessonsiktroduced ^at . Cl)anute are, in^general, not 
"flashy", or "clever"\ but. hondiescript gn^ medhodic^l . The creativity ^f 



the, authors may' have been limin^d.by tfiese 'constraints ; in'additlon, 



no 
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radical modifications to the baBi\ ISD sfc 
• their ostensible cost-consclousness\ 

••\l9\ 



arting point were permitted due to 

V 



_7 



\ 



4 • 
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' • Trjirfsfer of Programmed Instructiep Techniques to CBE '/ 

Inf luenpe ^ ' 

^ . The writing of CBE materials at Chanirte followed the traditional 

^ , - . ' t ' ' * ' ' • ^ 

approach toward les&on development*: ''that is, tlje pedagogical strategies 

~ of progMJnmed instruction (Pf) T Specifically, the structure of the lessons 

• A. • ^ ■ ■'■ '• ' ' - ' - ■ 

tendMftd be linear in construction with little branching to optional or 

» I ^^^^^ * • r 

. helptng' units. Also, the use of graphicafl displays .was somq^bat limited, 

• ' 7 

cpnsistent Oith the adoptio,n of the lean a|rproach to production; 
^ Tor' the mos[t 'p^rt , the indi<^adual lessors shared a common^ format 

r consisting ^of the f ollowing ^elements 
^, - 1. statement of* the objective; ' 

^"""^ 2, text portions with limited interaction; ^ ^ 

3, ^nd-<-&*^ lesson test (master validation exam or MVE) , 
^ A& mentione'd previously, ISp influence was m6«t strongly felt' in the 



construction of the. objecti^s and MVEls; however, there, was little control 
on their ^^art for th^ textual portions of the lesson. The ISD staff " 



me 



rely indicated that the text^tjie tailored 'to the subject matter of the 



lessons", but It was up -to the individual author to 'design the^e sections, 
. • * . * , ' ' f ' 

An additiojpal factoi: encouraging the utilization of t^e PI fo<j-mat ' 

may have been the limited knowledge that some* of the authors bad in the 

use of vardous TUTOR language commamis. Fewer aod simpler commands 

(i.e., -at-, -^^Iflte-) wcnild be needed for a linear lesson with little 

branching,'* Also, the limited use *of "hints'^ after an incorrect answdr in^ 

the programmed instruction approach may -have lent' itself readily tc^ ^ 

incor<)oration within the '.'bare-bones'" framework. There was little incentive 
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to provide extensive remediation sequences either ^fter an incorrect ^ ^ 

res^nse^ was ^iven or in a separate help unit, ' , . * 

• \ ^ ■ * ' ' , , • ' 

Discussion . * ^ ' ' 

' The above considerations have val^^dity, particularly in tenns'*bf • " 

proOTlfc^ a largd quantity of ins true tion J while minimizing costs. However 

it must l^e pointed out that the PL^TO system offers many^ capabilities for 

presetting Instruction in efficient and effective formats that Have not 

been (explored a^ (Jhanute* Individual branching for remediation or extra 

information has wide application for improving the ef fectiveAess of the 

instruction. Specific answer contingencies for pre-dete'rmined incorrect 

resjJonses can also -be used to provide the student with appropriate remedia 

tion. Immediate^ knowledge of results on end-of-lesson tests can giv^ the 

student helpful feedback on his level of protitiency. Hence, Chanute's 

4 

implementation may be described as. attempting to achieve cost~ef fectivenes 
by holding development* costs to a minimum^ rather than by ^^e^tly reducing 
the *t«raining needed by use of elaborate pedagogy or student management 

strategies/ , » j ' < 

Effect on Attitudes ^ ' 

' Special Purpose Vehicle Maintenance iristructors were asked for their 
views on the results of using, various fprms of instrucitional aids (i.e., 
visual, PLATO lessons, study, guides/workbooks'; lectyr^s,^ laboratory; and 
programmed instruction). Although the sui|yi/ey w^s limited to a small 



database (17 respondents), it may.be informative to consider their feeling 
on PI. Most of the respondents disagreed wltVi the statement "I get very • 
good results using programmed texts''^ (avei^age: 3.89 on a 0-9 very 



1 . 

For data and results of the questionna\*e , see Dallman, et al., 1977, 
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strongly disagree - very strongly ^agree scale). They were non-committal ' 
on the PLATO lessons (5.55). '.The standard deviations for the above items 
"were 2.14 and 2.15 respectively. , • 

Summary 

* It is of course obvious that few techniques such as extensive hint^ 

or remediatiotn sequences'* have been explored^in the Chanute lessons even* 

' 

though little extra effort would have be^n required. The heavy reliance 

1* ' ^ 

♦ on tried-and-true, eveti stereotyped, training techniques precluded this- 
utilization. The limited skill of the authors with the TUTOR program^iing 
language may have -been* another factor. However, the restrictive influence 

__of the programmed instruction format was probably instrumental in the 
limited employment of selected PLATO system capabilities. 



General Method of Le^on Production 



i 



A variety of organizational- structures have been usecL in I^LATO 
courseware development, and many groups have altered their structure in 
keeping with shiftingf goals at various points In thJ|^r^ project ' s histbry 
(^ijp^er, et al. , 1976). Such was also the case at Chanute. . In the early 
phase <rf the project, the authors worked independently on lessons in their 
own area of content expertise and instructional exprerience. In this mode/ 
thfere were no guidelj.nes for a commoj instructional- strategy and the 
authors had considerable freedom in developing the ^ individual styles.- 
Each had responsibility for content and lesson strategy as well as 
the actual TUTOR coding. This mode of operation can be considered the 
"individual approach". 

l!5rter, when the proj-ect goals changed to encompass develdptaent of ^CBE 

materials as part of an extensively revised curriculxim, it became obvious 

to the new project director (the head of the ISD staff) that a restructuring 

of the PLATO authoring group was required. In the first place, it became * 

necessary for some authors to develop materials for content units that they 

had not taught previously and in which they had had no special training. 

( 

Hence, their subject matter expertise^ was not broad enough to include 

/ 

these newlT assigned, areas. Secondly, the ISD'leader and his staff were 
n*t coiBfortable with/the instructional strategies evolved by the individual 
authors. .This concern was due in parf" to the inexperience of the ISD- 
team with CBE as a medium and, more importantly, because elaborate 
strategies, were not Required for the limited objectives and fundamental 
nature of the content planned for the new curripulum. The ISD group 



f 



wanted to work with "proven" strate-gies rather than "experimental" ones. 

' • / - . 

To overcome the twin deficiencies of specialized content e.xpertise 
and ^instructional design philosophy and \raining, £he project direc- 
tor instituted a "team" approach to development of the required 
materials. This approach is described In the fallowing section. It 
is contrasted with the individual approach, used extensively at an early 
st;age of the project and also concurrently with the team appi;oach in 
some instances. 

Team Approach ' • " 

For a major portion of the project effort at Chanute, a variety 
of team approaches' was .considered an optimum way to produc'e the most 

• efficient and maximum number of effective CBE lessons. Although, several 
authors had previously been instructors in the vehicle maintenance 

* course for the bulk of the period during which most of the PLATO l*es^ons 
were pi^aduced, ttie production managers for the PLATO lessons assumed, 
when it -came to curriculum development > that the autl^or^* themselves had 
little subject matter expertise and less instructional programming 
experience. An instructor may have /taught a specific block of instx^ic- 

♦ V ■ . ... 

tion several times, but he might know very- little about the material 
in other blocks- The authors were all recent instructors, but were 
ftot experts in material for all topics for v^ich they' wrote lesson^. 
In this v^ew, the main thing that the authors could provide was a 



facility ,| albeit a limited one, in the TUTOR programming 'Ian guga^e. 

to remedy this deficiency, the following tri-paftite organization 
was suggested for a typical production team. * ' * 

« 4 



1. An instructional designer^ ISD staff (part-time), 
, - who acted as a resource to:' 

»• a. ai4 authors who voluntarily sought him*' out; and 

b. 'review lessons and eliminate ndn-ess^ntial infor- 

^mation. • ; . - * 

2. A TUTOR author or programmer, PMTO sta^f (full-time). ' ' 

3. A subject-matter specialist (SMS), course staff (part- 
. • time), who acted as a resource tp: 

a. provice content for the programmer; and 

b. provide sub4ect matter expertise for the authors. 
In additioTl, other course instructors (not assigned to th^ project) 
were also consv^ted as well as reference books, old manuals, and other 
related materials. This organization wa'p necessitated by the btelief that 

{ 

one person alone could not contribute 411 the » expertise and experience 

needed for effective an^ efficient lesson#production. . 

^The operating principle Was that this method would succeed in pror 

ducing lesson material and in time the organlzation**could be modified* 

^he TUTOR au tit or c-ould gain some of the instructional proj^r amm^T s ^ 

expertise as each couj^ lekm from the other. In fact, it was believed 

that the author and instructional programmer could produce^le^ong inde-^ 

pendently of each other, but there would always be a need' for an SMS to 

assist both. There are several subject-matffer areas within the vehicle 

maintenance group that woul4 necessitate individual specialists in these 

areas on a continuous basis. In addition, the various SMS's could not - 

■ 

* m 

xleari^ the TUTOR language quickly enough to be efficient at lesson production 
iti their individual areas of content 'expertise. 



The lessons w^ere rfeviewed both whil^ in^progress and also after a 

* % ' * ^ 

^first draft wa^s completed. Usually an SMS examined it for accuracy of, 

contjSnt, although in an interview, * fqrme^ staff member commented that 

occasionally two SMS's would disagree on the ^^correctnels"* of a particul^ir 

point* The lessons were also%evi#wed by other members of the group; , 

^these "peer relklfi^s*^^ were viewed positively for their contribution to 

improving the- 6iat erials, according to former Chanute 'Authors. Reviews by 

"outsiders" t^,e. ; hTTC, AFHRL) were considered to, have little 'value , 

' / A - 

since'^ it -was believed that the external reviewers would not be aware of 

the problems and ^constraints under which the authors had to wo 

Independent Approach ^ * 

In HO sense were any of the authors- fully independent of other 

members or. thfe jChanute PLATO prbject or the vehicle course personnel. 

Subject matter specialists and/or other staff ' members had to be consulted 

from time to time for - assistance in coijng* or composing particular sec- 

tions. •However,* as the author gairied familiarity with the subject matter 

and the TUTOR language, there was a natural tendency to become less 
reliant" on others and more dependent on ones^J.f . An additional factor _ 

might have been the desire to increase the production rate df the 
lessons or even try out more individualized approaches to lesson construc- 
tion. " . . - 



Sui 



Since precise data on rates of lesson production w?re not kept, it is 
difficult to evaluate the fiiCccess of the group/ inc^vidual approach in 
cost-ef fectivenegs. Hoover , ^ according to fo|;mer authors at, Chanute who 
were interviewed,^ he group approach had some merit and' proved to be a 



# viable mode pf operation. In theory, it 'brought trogeth^r Ithe instruc- 

* tional designer, the computer programmer and the subject matter 

' - , ** ♦ . > ^ ' ' 

'specialist jto poq^l, their resources. In practice, not^ all membelrs '^yS^e 

team contributed equally, and thus one or more members had:,tp work 

somewhat independently in oVder to produoe a draft version of the lesson. 
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' , J Conclusions ' • • 

9 ' 

■ The instructional strategies aijd managerial techniques of the ISD 

\ . :> ' * ■* 

Staff were successful for increasing the lesson prroduction rate to a 
level, SO that materials for the entire vehicle repair course could be . < 
produced in the allotted time with^ the existing staff. 'Management by 
ISD caused a restructuring of tasks, procedures, and roles into a team- 
oriented production staff / Instructional design .by I'SD meant curtaiMng 

e^^erlmentfal approaches and adoption of standard strategies and techniques 

f 

By introducing the lean approach and enforcing the rule^ of "no new 
strategies, no new hardware", both a fldor aqd a celling were placed on 
the ^otentiaL sucdessf ulness of the final product. Specifically, 

because familiar, Pl-like traditional teaching techniques were employed, 

a., 

the "ingtruptionai materials h«id very little probability of' failing to 
teach adTequat^eflly and. were -also very unlikely to produce sweeping gains ' 
in perfiprmance^'' ret €(rit ion, -attitude, o^r time sayiags. 

"^Al^^ugh departures from the ISD techniques ^nd guidelines, can be 



found, in general the materials follow this coiiservative philosophy, 
•Though some decisions about less'on design seem to have been made on the 
basis of expedienc^r rather than I^D principles, the resulting training 
course Ti^s nevertheless met its objectives. 
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.Chapter 2: Computer-Aided Management at Chanute 




Abstract 

The -purpose of this study^ls* to examine one organizations efforts 
♦ • * -« ' * 

to reduce the time and cost while improving the effectiveness of various \ - 
management functions %y using the PLATO IV computer-based education 

(CBE) -system. The site selected for this^stuc}y Was Chanut^ Air Fotce , - 

{ - * ' 

Base, whose PLATO project staff made extensive use of- the data, collection. 

and 'retrieval , student records, and other available resources of the 

computer system. The 'data collected Included student test scores and 

specific responses to individual test questions.^ , The need for such data 

, collection and management of- student performance -^ores has been previously 

pointed out.*^ This report was submitted in dra^t form to Chanut.e PLATO. 

staff for their comments an4> suggestions. 
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* ^ Introduction ' * 

■ 

■The purpose of this study is to examine one organization's efforts 
to reduce the time and cost of various managemenl: functions in a military 
training environment by using the PLATO IV Computer-based education (CBE) 
^system. Tl\e site selected fdr* this study was Chanute Air Force Base, 
whose PLATO'proj^ct' staff made extensive use of .the data collection^ /> 
-student records, and 'other software resources o'f the CBE system. ^^^jb ' ^ 
/ data collected included" student test scores and specific responses to-; 
individual questions. The need for such data collection and management^ 
of student performance scores has been prev:^ously pointed 'out (Dallman, 
1975a.' p. 53). / : ■ . \ . ' ' 

. 'In order to gather data for the Service Test of PLATO IV at Chanute, 
the Training Research Appli<:ations Branch (XRAB) and the "instructor per- 
sonnel in the tai'get ^courses administered pre- aiod post-tests, recorded 
studfent block grades, remedial instruction time and w^shback rates (Dallman, 
1974, p. 4)»» Sinte the completion of the data collection for the Service 
TQst, similar , course management activities' have been' carried on by the 
, reorga^nized PLATO authoring staff* in cooperation with coi^rse ins true t^Drs • 

The capabilities of the, system haye been used in Order to make data 
^i^Pollectiori aild management more efficient and to ease the burden of th'e 
instructors from f ime-^hsuming tasks. ' 

, Tij^' IjjBQ^^fJSt of this report is to describe the management applications 
of PLATO ^"t ,Chatiute. Chanute 's techniques are not presented as an optimal 
way to utilize all system data-related management capabilities but as an 
example of one site's initial efforts in this area,' * ' ' 
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Selected Capab'Ult lea'^the PLATO IV System . , 

The PLATO IV CBE system lias many capabilities for data collection and 
retrieval that can- be employed by the 'average user. Ll^e any advanced 
computeif system, it can manipulate dat?abases to provid^^.yBK'iiiation If or 
decision-making or record-keepings It is especially Qonvenien^t to use 
the system to monitor or^manipulat^/ information that is generated by 
on-line student use (e.g., test scores, r-€sponses to interaction, accessing 
of "help" secfuences, etc.)» The collected datar-^are available when heeded 
by the instructor to monitor the student's progress and/or detect 'the n^ed 
for remediation. 

Figure 1: MVE Ti-mes and Grades, 
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Utilization of Sygtem Management Capabilities 

Monitoring Student ^^s Progress ^ , 

Data, including scopes on te^xs and individual responses to' test 
items, have been collected, for Students in the Special purpose Vehicle 
courser In addition, d^ta have beea collected for students in the General 
Purpose Veh:y6le course. . For the latter copr^jj^trhe data were of a 
longitudir|al pr chronological variety and •also included course completion 
times and bi-weekly tfests known as block ^x^s. . \ 

Chanute PLATO author staff created programs. to collect and display 
a variety of student performance data. In the on-line student ros-ter^ 
(maintained on the PLATO system), the student is listed by last name, and 
th^*^cores on .the master validation exams (MVE*s) are recorded along with • 
the amount of time tViat the student nee'ded to finish each-test (Figure 1). 
The, number of minutes needed for, completion of the MVE is important since 
it probably indicates the student's level of knowledge and* rate of progress 
2II a&^"showin§^ the instructor vhiclj/ student^s^re having difficulty, . 



as weJ 



with the material. Xhat is, a long completion time and a' low score may 
f 

indicate the need for remediation or* at least consultation with the 

student to ascertain a possible problem with comprehension-, while a fast 

completion time coupled w^th a high score' suggests some • prof iciency in that* 

area. ^ • . ' % , 

♦ 

' While collecting the studeat responses and grading the test, the 
computer ianplements the scoi^in^ formula or » "correction factor" in use at 
Chanute (total score = [number correc [1/4 ^ number wrong]). Thus, the 

mathematical grading functions thair would have to be performed manually by 
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the instruDtor can now be performed much more rapidly and . accurately by 
the tomputer. In^^Blition, student data can be stored for selective 



retrieval by the instructor or student: the instructor can use the 

data to determine the student's level of proficieney and the student can 

- . ^ '. 

monitor his own progress and/or need for remediation. In addition, the 



test itself may need to be modified .based on the results of stiW^nt data 
collection. That process of test modification is discussed in a later 
section . 

Information Storage and Retrieval ' . 

' . The data that are collected are easily accessible to instructors 

and/or their supervisors. The options available in the-,Chanute 
-information retrieval package are listed on the^ndex page of the pro- 

^gram (Figure 2).* A permanent record of the cJesired data can be produced 
by means of the Varian , copie r (a, device attached to a* PLAI^^^terminal 
to* record images from the screen display) in the autho.rs' office at 
Chanute» The number of instructor and/or supery^sor comments relating' 
to questions on the tests is listed (Figure 3). The comments can also, 
be accessed by the instructor or supervisor from the same "information 
retrieval package. 

The inf6rmation re(^uired by Air Force form 668 for recording 
test scores has bee^ placed on-line (Figure 4) . Tke responses from 

• thre students are now tecorded automatically after they< take a test 
and the jEorm is filled in without' the instructor's intervent ioflij ^ The» 
effort needed for what was once a '*time consuming clerical function" 
(Dallman, 1975, -p. 53) has been reduced from about an hour to ' 
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Figure 2: ' Test Times and Grades 
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Figure 3: Instructor/Supervisor Comments for each Test 
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•Figure^ 4-; — Qn-Line AF Form 668 

Class // 760414ad 11 Students' 
Test // tll3 • . ' 
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10-20 minutes.^ Since approximately 7 tests ^are given during the 6-week 

course by each of the li instructors and VI of the 29 tests included 

in the course have thus far been pjit on-line, the time saved can be 

calculated as follows: ^ 

7 tests^ ^ 12 tests ^ 50 mln > ^ 1 hr ^ , , / ' 

^ "^TTTT — ^ ~^ ^ T7^ — ~ ^ li instructors ■ 



6 we^s • 29 tests 1 avg test 60 min 

4.5 instructor-hours/week 
Thus, a total of more than 4 instructor— hours 'per week has been saved. 
If all tests were on-line, a total savings of approximately 9 man-hours 
per week could be projected. The net -savings in dollars is difficult to* 
calculate since it is not known how many hours were required to produce 
the original data collection lessons. Nevertheless, the benefits resulting 
from the utilization of- such lessQps are significant and widely applicable. 

1 V \. ^ 

,The larger saving was estimated by the instructors, while the more 

conservative estimate is found in the PLATO IV Data Automation Requirement 
(DAR) endorsed Uy Major General Edwin W. Robertson II, on 9Aug76. Depart- 
ment of the Air Froce, Headquarters Chanute Technical Training Center <ATC) , 
Chanut;e Ai^ F^rce Base, Illinois 61868. Sec. 2.g. (3)w 
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; . . Specific Applications of Qata Management 

^ • ' * 

Author Mde:' POI Editor 

— ' — ~ ■ 'J 

At one point In the project, it was believed that plans of instruction^ 
(POl's) outlines for'teachlng a particular segment 'of material, were r 
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'of material, were r*\ ^- 

subject to a series of rapid, yet relatively minor format changes. Although^ ' \ 
the re-typing and reprinting times could bat4ly k^ep up with the changes, 
the obligation, to h^e an up-to-date copy of each POI"* i:emaln€*. In an^ * 
effort to satisfy this requirement, work on an on-line programming routine 
(an "editor") to effect these changes was commenced. One of the Chanute 
author staff was given the task of creating such an, editor which would 
allow POI*s to be changeii. with minjlmal effort. 

A study of the potential cost' savings of the POI editor was Included as 
part of the contract^-^or^which this report is being produced (CERL, 1975, 
p. 38; also Daiiiikrfv* 1975b*, p^ 3>-. However, aft^r the form of the .contract 
was finalized, the Chanute development effort was abandoned. Undej normal 
conditions, the POI's are only changed on the av^f^e of once; a year, and 
consequently the need f^r such an on-line capability would be limited. 
Though it was decided that work on the editor be suspended before a fully 
operational routine was achieved, -it should be noted that enough Work had^. 
been done so that* it was possible for tljis evaluat,or» to prepare a working 
editor after some minor empirical testing. 

Student Mode . * ' 

Storage of student answers . An expanded version of a studefit test 
xlata collection package written at Chanute includes a lasting of all 
, responses for each cjftestion on a test (Figure 5). According to Air Force 

39 ' » 



Figure 5: . Listing of Test Item Re^potises 
Class// 760203ac ' ^ 



Class average 91% 
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regulations, thd J[nstructor and his supervisor are re<|uired to give a , . 
written explaaatioti for any question on a test that is missed by all the' 
students, none of the students, or answere^ with, the same response by all 
' thfe students. When the PLATO system is used for .collection of indi^Kdual 

• ■ ... • ' ■ • ' . ' ' 

item- r^sj^nses, . the instructor and his supervisor -^haye all' the information^ 
readily)" avaiiali|| to make a dl^gnosis.of *phe above problems. ^ 

r , ^ 

Data gathered in an analysis .of materials pfoduced/at Chanute als^^ 
^ suggested that the test ^em responses were ?n' f^act used for ^nM^tlye " 



. % 

modifications: that is# the test qwestions we 



re^evifeed to make^^^^^*^ 



easier to understand. Fot further informatiqo^ see the section on "Tests", 
and Appendix II in Klecka' (1977). - 

fStorage of instructor cijmtapnts. The recording and storage -of the* 
instructor cpinnents/have also be^n 'automat^ (Flgtire 6). After noting one / 
crt more ijicidents requiri^ Analysis atid cotnaentary, the instructor can , ^ 
advance an explanation for the a cfctial i student responses. Then* the super- 
^sor» before making his' own comments, qan review ^e actual question<s) 
along witlf^'^e^oSserved breakdown of the responses. ' These remarks are 

•." ^'.-T — ■■■ \ ^ 

then stored^ and are madd accessibl'e to- botb instructors and other super- 
* visors. Both sets of commeifts can be* use'd to mcdify-^^e test question(s) 
» if 'such action Is. deemed appropriate. \ 

All tne dgita niScess^ry for making a cc^pament;;, are readily at hand. .The 
comments tnMiselyes* take the form of a brLef phrase jor two giving a 
^^^^^^ggested ex^ 



^ for the' .par^icul^ pl^npmenon" that pc^rred':^ Much 
titue is sa^ed in making these <iotnmdrfts since the individual;**test? pap6rs 
.peed npt be drawn .frpm one file and the comments selected from another file. 
4 Ail can be vj.ewed in a f^w m*fi>utBS without the iifttructor ileaying the / 
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Figure 6:* Orr-Line File of ^^^tfuctor/Supervisor Coraments 

" Class number* / 760204 - 
• % Instructor / blake 
Date / 05lQ5ne 

test was reviewed and considered 
'valid. 60% of the \ ^ • 
^ class miss^ed question, '20 seems 
• ^heywere confused with - . ^ ' 

' the ^oil^ce of the oil ' 
flow i;^ther ^an the^^th .of 
t)il flow, test was 

critiqued in cla^s.. I^^v . * > 



Class number / 7-6620* - ' 
Supervisor- / rogers * ' ' ■ 

Date / ObfObflb / " "'^ 

material pertaining ta q|^stioni2Q - 
is well covered in piato I " ^ , 

lessofts ^question ,is va^idT j 
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terminal. The savings in time and effort of limited manf^ower is consider- 
able since this review is a mandatory process, But it is not easy to estimate 
preQisely how cost effective the new computerized system' is. FurthenTMDte , 
the review pro^cess has been improved, in the opinion of. the Chanute instruc- 
tor staff, because the presence of both data and ccmments in an easily 
acce$sible« format encourages and»'f adilitates a more thorough evaluation. 

* Retrieval of data a'nd'use. According to Chanute staff -who were inter- 



viewed, the instructors have accepti^ed computerized data collection. At first, 
-when they di'd not adequately understand the; sytem or perhaps felt their jobs 
* were threatened by it ,^ the' H^tructors apparently were not very enthusiastic. 

However, after t\i^ saw how their time could be bett^ spent ^n individ^l- 
'ized help for their students and how much more ac;:urate the PLATO IV system 



could be, their dwn experience cbnvinced them of . its effectiveness. 

The reliability ^of the old manual system of recording data^'ls not 
known* precisely. ' However, it was widely regarded by the Chanute sta?f is a 
v^ry lal>orious pretedure^ the results of block tests with 30 questions each 
and 5 categories of^^ossible answers ("'a'^ through "e")' for 11 students per 
^class^ad to, be transcribe^ Alternatively, when the instructors interacted 
directly with the computer,* t*ie neces^ry procedures for utilizing the data 

collection routines were seen^to- be quite straightforward 'anti less tedious 

' 2 ' 

than the manual method of :Pns^rting numbers into a long AF form. 

On-line lessons were also used to maintain an up-dated sta^uff report 

on courses undergoing revfsiot^'by the Instructional System Development (ISD) 



' 2 ^ ' 

^ This was reporCetl by PLATO project authors who trained the instructor 

aff in the use of these routines and also provided a special lesson 

help'*) as an on-line aid to the various data collection {Procedures that 

wS¥e available- • ^ * ^ \ 

Er|c . • ' . 43 ^ 
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group (Flgtire 7) • Projected dates were given for the work, or the course 
m . ^ ... ^ ' 

was marked as completed when it was:done, Thus^a continually updated 

\ 

picture of ^rie status of the course could be maintained and ^easily accessed. 
The instructors concerned could be kept abreast of any developmehts related 
to their courses/ ' \ ^ ^ ' 

A pre-test has been put on-lin^ so that prospective students can be • 
rdnkeU approximately according to their probable achievement in technical 
are^s ("clwi93")* At'®ne time a paper version of the pre-test was used for 
five courses (25-30 ^udents ^jer month) in the Department of Weapons System 



Support Training to^^lve tHe, instructors an idea of what types of students 
would comprise each new class. The test seemed to work well for students 
%n the vehicfLe training courses, J^t for students in other disciplines such 



as' airframe maintenance ^ it did npt ^a^qu'ateely Reflect the likelihpod 



of their success. ^ 



The pre-t^t shpws a definite vehicy^ar Orientation, and this may have 
been a cause of its limited v^ltdity^ yhen used ^fo^s*4^t her courses. The 
pre-test is now ad|p:|pistered obXy to vehicle maintenance st|(dents. Here 
too, the scores are '^to^reji* ajfii'^eferenced liy- surname vW%Lle the students ar<^ 



given an aptitude level'^^lfpi: ^he*" fc^lowln^ areas: . general, mathematics, 
vocabulary, easy* aT\d hard^iechn*ical . . * ^ 

In all these areiis lof test admini^ration, the di"f ferent toucines 
collet and store the. data unt?l'the result^ ar& needed*, In addition, the 
file's (iie., stor «Tg » * locations on computer ' a ^jaghetic disk) can ^e cleared 



out to Inake room fof new. list^of students when the instructors decide t 



o 



update th^m. The procedures for entering a* j|ew c|pss and reiniMi.alizing the 
data^s^i^ge bank appear b^ simple enough tasks so that* the instructors 
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^ Figure 7: Status of Courses under Revision * i 
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1. 


2ASR4725I -81 


Sep ^76 . 


104 hrs 


* - 


2. 


2ASR4725iB-3 , 


May 76 


80 ♦hrs 




3. • 


2ASR47251C-1 


Dec 76 


0 hrs 




4. 


3ABR47230 


Sep 76 » . 


550 hrs 




3. 


3ABR47231A , * 


Comp 


640 hrs 




6. 


3ABR47231B 


Comp 


480 hrs 




7. 


3ABR4itO0C 


Comp 


360 hrs 




8. 


'3AZR47251B-1 


Jun 76 


80 hrs 




9. 


3AZR47252 -1 


Dec 76 


240* hr& 
< • 




10. 


3AZR47252 -2 


Dec 76 ( 


104 ,hrs 




11. 
12. 


3ftR47i52 -3' 
3AZR47252 -4 


Dec 76 V 
Dec 76 


120 hrs' 
40 hrs 




13. 


3AZR47252 -5 


Oct 76 


40 hrs 




14. 


4AST47250 -1 


.Sep 76 


80 Hrs 


15. 


4AST47250 -2 


. Dec 76 


0 hrs 


< 
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, ^ Explanation 

AbIT = airman -basic resident (a beginning 

course l,ike the GPV and SPV courses). 

ASR = temporary course for 3, 5, or 7 skill 
^ levels (journeyman or advanced). 

AZR * permanent course for 3, 5, or 7 skill 
levels (journeyman -or advanced). 
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do not^^eed a$sist;atice from the project staff, atcbrdlng to the staff 
members' consulted in an interview. In t\iis application, the computef 

^ \ \ ^ , ' 

al'lows a new service' to J^€L..#ftSvided expeditiously. Unfortunately, no 
cost savings is calculable. Presumably, the effectiveness of the instruc- 

ft 

tors is enhanced by early diagnos'is of students' potential problems. . 

-» * 
The above views are based on lengthy interviews with the instructor staitf; 

at Chanut^^ ' ^ 



J 



/ 
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^ * Conclusions , 



During the planning stages of the projJbt, Chanute ^aff found only ' 
limited management applications for the PLATO system. The capabilities 
of th^ computer became more significant* for handling the substantial 
amounts of data collected during the formative development period. - As the 
pjroject matured, the computer as,|umed a larger role as thev data-keeper for 
managlQlf st^udents or l^l^onnatlon about students, 

A precise cost/benefit analysis Is difficult because many computer 
applications allowed an inqjroved quality to be achleyed for a task which 
had no crlterlai Fot example, the existence of readily available data 
collection and managemetit routines permitted and encouraged more careful 
analyses before making decisions. Although "better decisions" are 
certainly a desirable goal, it is difficult to determine' a reasonable cost 
for them in concrete, dollars-and-cents terms. In other cases, there is no 
clear-cut way to jboi5jj^5i9i4,)]^e^ra.QCU]rj^y of computer-tabulated data against 
previously unaudited entries or jto^assl^ an economic value td the 
(presumably) more a9curate data. / ^ • 

^l^any event, the use of the PLATO system at Chanute for testing and 

data management was significant for the following reasons: 

/ * <■ "> 

1. it freed the Instructors from tedious, 'tlme-corisumlng 
tasks to give more attention to helping individual 
students; ^ « 

'2. it reportedly Improved the accuracy in, recording* 
V specific 'responses to questions as'^well as the test 
scores themselves; 

3. it allowed rapid and convenient access to relevant data 
for instructors arid their supervisors who bad need of 
the InformaMbn;^ 

4. it made retrieval of the actual test items as well* as 
the results of the tests readily accessible on-line. '* 



^ .1. 



Summary Tables 

For the convenience of the reader, observations related to the 
detailed data presented in t-he tables are given below. 

1. Th^nmnber of lessons used for d^a collection and 
testing of students as well as the number of times . 
they are accessed Indicates such use was of substan- 
tial in^ortanoe to the operations of ^haifute: from 

4 Octpber 8, 1975 until October 1, 1976^ these programs 

/were (ised for over 800 hours. 

2. The amount of time originally expended by the sluthors 
in the preparation of these programs is difficult to 
determine since exa^ figures on production rates 
were not kept; however,^ ^ 

'3. The Chanute Haff indicated that approximately one 
nt>nth was spent preparing the revised forms of those 
programs wit^h old and new versions described in the 
appendices^ ^ 



A. 



Student response data h^ given au||^ors guidance for 
modifying and improving tests. 

5. The Chanute author staff is continually' -working to 
improve th^^^ta collection routines by streamlining 
the programs to enable* more 'efficient coding and 

* .there*by reduc:e* ECS^sage. 

6. Efforts to improve the data collection routines indi- 
cate tl^at these programs are considered useful for*- 
the instructors and studients, as well as the, author 
staff. j 



/ 



42 



Tables of Selected Test and Da^a Management Files . 



V 



Igt roductiott ' 
T 

The following are descriptions of PLATO programs used for the 

collection^of student responses to questions and scores on tests at Chanute 

as well as sample tests themselves and other data collection programs •/ 

Some comments on them are given, as well as the amount of space (ECS^ 
N 7 . 

needed, a factor involved in their utilization. FHarthermore , in several 
cases, there is an old version of a program that has been or is being 
phased out by a newer improved version requiring less space. Data are 
given fo^ both versions in those*^ cases* Readings were taken on September 15, 
*1976 except where another^ date i*s indicated. 
Explanation of Terms 

1. High ECS = over 10,000 words . ' ' 

A large space i;equiremeht would tend to make it diffi-^ 
cult to use the lesson-between 10:00, a.m. and 3:00 p.m. 
on weekdays when the system-wide demands for ECS are' 
greatest . ' ' . - 

'I . . 

2. Usage : number of times lesson has been accessed and the 
amount of time spent in the lesson since it was created 
or since October 8, 1975 when accounts dkta keeping was 
begun. 

3. Non-Chanute ; usage of lessons at other sites such as 
Parkland College in Champai'gn. 
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Program:. * ^ spshred668 (formerly"' "cha25*') 

• ECS = 8dfe5 • ' (formerly 11595) 

Subj^ect matter: AF form 668 - item analysis for test evaluation, 

Usa ge: <^ Uses • • Hours 

\ Total 37,52 502.8 

' Chanute 3138, 464.0 

Non-Chanute 219 38 . 8 _ 

(Figures on usage as of September 20, 1976.) 



Program: cha95 - , . • ' 

ECS » 6392 * ' . 

Subject matterl MVE results ^iin datafile: daily appraisal of end* 
of lesson test results. 

Usage: * ' Uses Hours 

Total • 274 ' 16.2 

Chanute 55 ,2.2 

' Non-Chanute 219 14.0 



Program: chal39 

ECS = 7163 

Subject matter: Block and pre-test scores and analysis pre- vs. post 
test scores. 

Options: ' 1. update file 

2. student data *and test data 

3. block averages 

Obsexrvation: *rhe figures for **chal39" are as of August 1, 1976. 

Deletion' of this program occurred after that date 
since it had had limited use, according to Chanute 
' staff who were consulted on September J7, 1976/ 

Usage;' Uses Hours 

Total / 165 . 6.2 - 

Chanute 4 . 1 



^ Non-than^e 161 6.1 

■ 5G 
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Table 2: Student Management and Instructor Help ^ 

« 

» 

Program: • ^ charich (formerly "cha39") 

ECS = 1500 (formerly 3252) 

. Si&ject matter: _ Enrichment, lesson list ^ 

Observation: ' The enrichment lessons were used when a student 

.con5)leted the assigned lesson before thpitime period 
was up. The purpose, according to one staff member, 
was managing students who might otherwise have wan- 
dered around the room and interfered with other 
students still working. 



Usage ^ ' * Uses Hours 

llfotal 22A4 60.9 

'Chanute 2194 57.2 

Non-Chanute ' . 50 3.7 



Program: cha93 ' ^ s. \-/ 

ECS « 5334 ^ . ' 

Subjecf matter: Pretest to determine student aptitudes in var 

Options: 7^ 1. all studen'ts* scores 1 

; ' '2. weak students'- scores by 4rea 

3. total test analysis 

4. individual students 

N 

» Usage: Uses • Hours 

•:^tal ' 719 191.7 ■ 

^ Chanute ' 47A 180.7 

, . Nori-Chanute 241 11.0 




c 

am: ^*-~^< 



Program: ^^-^-^chaSSg^ - 
ECS = 1500 



Stfbject mitfter: Special^JJeijiciefi-- ?0I Form with a sample lesson plan. 



~ 4 



Observation: ^ Examination of this routine revealed that no special 



editor w^s, needed. ^ 



4 
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Table 2 con't 



Program: chahelp 

,ECS = 2914 

Subject matter: Orientation to PLATO use for instruct^or s. 

"Options: (table of contents page) 

1. introduction 

• '2. .writing test conments \ 

3. entering students in a course 

4. deleting students from a course' 

5. assigning modules^ 
^ 6. lesion sequencing 

7. assigning block or shred tests 
8^ locating student statistics (i.e., grades) 
» 9. notefiles (instructor and student) 



Usage Uses Uses Hours 

Total 155 7.9 (all ]by Chanute as of 

October 20, 1976) 



Program: chal32 

ECS = 4147 

Subject- matter: Data on courses and ISD work. 

Options: 1. data by branch. 

2. data by type ofNcourse ~ 

3. percent completed 

4. completed courses 

5. all courses 

6. add new courses 

7. scheduled courses 

8. overdue courses and courses due next 60 days 
Usage: ^ , Uses Hours 



Total 344 17.8 

Chanute -^191 11.3 

Non-Chanute 153 6.5 
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Table 3r Tests 



Program: - 
Course : 

Subject matter: 



blocktOOl 
ECS = 3250 

'SPV 



Subject : block x 



Observation : 



Usage : 



''^aiaple of a t}^ti.cal ' block test (there were approximately 
15 of these block' tests on-line as of November 3, 1976). 
Questions on a variety of topics which the student had 
to learn by self-study outside the classroom. 

t 

Very easy for instructors/supervisors to read other t 
comments and write their ovm in this l^esson, even tor 
someone with limited experience. on the PLATO System. 



Total 



Uses 
14' 



Hours 



0,7 (all by. Chanute) 



Program:^ 
Subject matter: 
Observation : 



Usage : 



shred511a f . 

ECS = 3746' ' , 

Block test -for common course segmertt (this is- a- sample 
of 23 block tests, now on-line). 

Fotl^ block exams for the portion of the course 'common 
to the four shred- (i.e., the portion of the SPV for 
^which the PLATO lesso^ materials were developed) are ^ 
on-line and 'have been used. Following the six week 
common course segrftent , the students were given instruc- 
' tion in their owr^ specialty in each of the^four shreds.^ 
• Block exams during this specialized training period • 
are called **shred exams'*. At th^ time of' the inter- 
view, all six of the "D" shred block ■jexams were on-line, 
and in use. They *had been put up gradually, or at 
l^&3t not simultaneously.' The "B" and. "C" s^hreds 
each had two, exams to go on-line right away and two 
more coming soon. The "A" shred had not been revised 
by ISD, and hence its' block exam§ would npt be on-line. 



» Total 



,Uses 
8'1 



Hours 



7.2 . (all by Chanute) 



. V 



r ' 

Table 4;- Test Results and^NtAnalysis 



Pfogram:, spgrades (formerly "cha90*') 

ECS = 2524 (formerly = 3267) ^ 

Subjecj^ Wtter: Block test results (names, 'items mis^sed^ score). 

Options: 1. see data by, class ^ ^ . * 

.2. see data by* test : - 

3. see data by student • • 

4. zero data 

, 5. 'update * h 

Observation:, The above optiws show how easy it was to exa^nine 
and reinitialize the r^ording of data, even for a 
novice .instructor. • 

Usage: ' Uses Hours , 

Total • 496 16.7 /• 

Chapute 276 - 9.8 

Non-Chanute 220 ^ 6.9 
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. ^ Dallman, Brjan . P^ATO IV ^progress report for Chanute test site, 
- , lNov - 30Nov74 , |i 
Dallman,' Brian. » Instrijct^nal materials d e ve 1 o pmd^/a t the Chanute 
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Chapter 3: A Survey,. "of 4hanute Lesson Dj^velopment Techniques' 

■m ' ■ . . . ■ ' 



Abstract > ♦ , y ' .* . , 



A survey o'jpM^he^process of lesson revision and development at 

■'^ A ■ \ ^m- \ 

* Chonute AFB-was considered impcirtaQt to. determine ij^s effects on the 



lessons and the pedagC^gical impact of the*^omputer-based education 
roateri-als. €ight representative lessons were selectc3%by a Chanute 
evaluator. In the course of revision, these lessons wfe*re/ subjected to 



f subje 



a*s^ies ©f .modifications by Chanute staf f 'members t^^ ix^suTM^their 
validation -when used by miHtary, students . -^h MTC eyaludtor prepared an 
in-deptjl anal^si? of the "before"-and "after" wersions of each lesson, 

.to aisc^rtaiti the cH^n^ges and^ t^-heir impact, Evaluator^ at Chanute were 
• given ^r^Yt ve«rsions of tfie teport for-»their cijiument is which were included 

*iT^^^he f*tnal version, ' 
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Introduction 



Backgreund and QbJectives ^ i' 

This is a study of tljie changes made between two series of Chariute • 
lessons: an early and a later series. The first series consisted af 



, lessons before validation had been atbempted^, but after trial stiilents * • 
had been r6n and after initial lesson di|felopment had been hsuLted. The 
second series iqll^ded these same les^Hj after they had undergone revi- 



sion, whejre necessiry, based on student data. The revisions were B^de 
by'Chanute ^staff to Increase the likelihood that they would validate: 
i.e., thart-90% of ^he students would get 80% or better oir the end of 
lesson test. ^ ^ 

. The objectives -of this report are two7fold: 

1) to ekamlne the lessons to asgertaln what changes 
have- been made, and « ' ' 

2) tfto determine the Impact of the changes on student 

perfbrmance . ^ * ' , * > 

( • • / ' > ^ ■ 

"A more thorough dlkcusslon •cff the individual lessons can be found in 

• ' * J, 



c An Overview of Chariute Lessons > MTC repprt #10. The aut^^^assumes that ^ 
'the reader has re^d or has available that report since It will, be referred 
to ooca-^pnally (ly^reafter caUed the Overview). . 
Method of Approacn ' * 



These analyses compare two versions of eight lessons; one version 
* ear^y in its developmental process and another version which had 'been . , , 

modified over approximately 9 months, including 6 moriths use l|y a total 

' ' ■ ' 'i 

of about 150 student*. » tSee Appendix I for details.) The ways in which . ; 
# .. . * 

each version differed were noted and evaluated. By this procedure ' some ^ ^ 

i ' V ' , y 1 f 

# ' • • ' ' \ . • 
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• . . . . ■ ' - 

insight can be gained into the revision process used to improve the 

. X »- : ■ 

probability pf validation and into the process of lesson development 
^ as practiced in the Chanute PLATO prcyject.* ' . • 

The type and quantity o£ changes, were by no means uniform*. In 
some (^tses^the modifications were of tniiioif^impo^rtance^perhaPs even • - 
^negligible*. In'others there were significant structural and pedagogical 

• - ' 

remodelings. As roight be expected, the latter will be, explored in 

• °' . • ' • . - 

' ' greater ^detail since. they have had a greater impace on the achievement / 
',of lesson objectives. ^ Throughout this chapter,* the lessons' will be • 
referred to by a name and number, e,g., "PJO" (1) "indicates the pre- 
validation version whi]?e J'PTO" (2) indicates the post-validation version/' 
the lesson on t;he. Power Take-off unit. . *^ 

As part of tl)e formative development of this report, drdft cojjLeie " . , 

\ ^ 

were submitted to the evaluators arid authors at Chanute. Addit iot\^ ^d 
" corrections were included in the report at appropriate peaces, 
addition the Chanute evaluatbr's comments are included at the end of 

4- • 

this report. ^ . ^ ^ 
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^tructu/ial Analysis ' 

. . . _ t . . 

f 

General Comments 

In ail lessons >t>ut ^iPT0"(2), the structural chang2^ did not warrant 

a new detailed -iiagyanr^i. e.., flowchart). These diagrams (not included *with 

this report) were-used extensively for thl>s and other Chanute rep<rt:ts, , 

notably the Overview . -For more lietails on this and other subjects 

> * ' . •* > . 

mentioned h^e, that report should be consulted. , 

One observation is in .order at this point. The lessons In the 
second series share an indexed linear format (except^or "emiQj^on'^(2) , ' 
which is" basically non- indexed) . Tl^ modular approach is a later additjion 
to tKe instruct i^onal design of the l es sons. In most cases, the lessons 
are now .organized in such a way that the .student can choose when he is k-o 
do each Section. This .is an important move toward increasing the -j ^ 

individualization of the Wesson. - : • ^ 

Specific ' CoTTTinents - ^ * j • 

"PT0"(1) and (2) . Since the copy of **PT0"(1) available to the 

i ' . . i 

reviewer was not comple^, the revised form, **?T0" (2) ,' was analyzed in i some 
detail. More extensive temarks may be made about this lesson than aboiit 
the other-s in this series becfause of the lack of previd^ commentary. 
"Starter"(l) -and (2) ; The lesson "8tarter"<l> was also available 

— fi , . • . ; .[ 

only iri ^n incomplete f og|^* - The later** version, "starter**(2) , While 
differing in many aspedts fa|mits predecesso^^^^owed onl^ minor strud- 



rwiL i 

w 



i 

i 

Lc; 



• tural differences. Cfopsequ^jtly these did not wiirrant a new scbenlatic 

they will betmentioned in the discussion of organization ("intta-lessoii*') . 

N i * 

"Starter'*(l) had only a superstructure which was not filled. In ,' and < 

X *^P^0"(1) displailgd the beginnings of a whole lesson; thesje did not possess 

•ERIC • 




the usual characteristics,' i.^. obj«4-tive, ind^x,*end of lesson test, 
etc. to make it even eligible for' analysis. , * » \ 
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Inter-lesson 

» * » 

"By-pass test" option . The most significant organizational change in 

- . ' ' ^ 1 . 

these , lessons is the elimination in all but one of the Icssbns, of the 

*optfbn ta t*ake the end of lesson, tel^ immediately upon^ entering the 
lesson. This "by-pass" test option 'is now available only in "diesel"(^), 

'and its availability may be an ovefsigjj^^rather than a deliberate design 

feature- Consequently the student must now go through the whole lesion 

before he is 'allcwed^to take the end of lesson (i.e., validation) test. 
'Acp^r^ng to Chanute staff, this change was' or iginally made tp 

furnish additi-onal valiiiation data. It was. allowed to remain ^fter 

' valiration betause it was. decided that there was no advantage to ^saving 
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the time of the good student in the group-pa^d usage. Also, students 

selerted tihe by-pass option ^i^epeatedl^ to a%oid*takin^ the whole 

I'esson; if they did not pass, they were cued to be especially attentive' ' 
^ * . 

to certain portions of the lesson and may have neglected other area*s as 
a consequence. The option was removed, rftther than re-coded to allow 
only one attempt at the end of lesson test. Apparently few students 
passed the test as an option. Thus it was felt that the students' 
time would be better spent .in Agoing through the lesson rather than 
making repeated attempts at the by-pass test. \ 

"Teaching points" . A noteworthy addition to eacTi lesson is the • 
presence of a detailed outline available to an author (NOT a student) 
from the page stating the objective. J This outline is much more detailed 
than the brief one found in rhe^trtfex to the lesspns*^ In only two 
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•lessons ("driveshaf t" and 'meet ricity*') are directions given for • *- 

• 1- 

accessing' the teaching points; in the others it Was discovered by- an 
. examination of the coding. The reason Chanute ISD (instructional^ 
system development) staff stated for not allowing access of the^e ^ ^ 
"points" was that "the student will eventually see the information in 
the" teaching points when he takes the lesso^/' The reviewer feels that 
the iisefulness of the teaching ppints to the student- may substantia ^ 
and should at le^^t be investigated. (See Appendix IV and "General 
comments: Components" of the overview.) ^ - n 

.' Other new material . In some cases information is presented in the 
second series of lessons tha|>%does not appear in the earlier versions. 
DcU^f.ils of such ocpurrence9Mwili be found in the. section on "new*|f[ateYial" . 
The informatibn presented in^that section was not felt to be significant 
enough to change the organization of the le9Son in any substantial way, 
and thus is not included in the' following discussion ("General comments: • 
Intra-lesson") . , , - - 

Intra-lesson ' / , . » 

A few specifics on significant* organizational chafes: ^ 
"starter" (2) 

The, large help section- of "starter"(l) has been broken up 
-into convenifent-sized sub-section^. Tihese 'iare listed 
an index page and" can nt5w be chosen at wljj. by the students. 
They can go directly to a specific topic and need not 
wander around in one large section as before." 

.--.^ The individual section pre-tests (i.e., Stfiction bypass - ^ 

te3ts) have become end* of section tests', and th'ere is no 
o()tion available >for taking them in lieu of going through 
the particular section. ^ 



Of the effeht lessons examined, oniy"starter']-(li|^ontained. 
such sfection pre-tfests. 
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"transmission" (2) X. 

The Index page •is more useful: it. now appears at the 
beginning of the lesson , -instead of th^ end as in "trans- 
mission" (1). Also the students h^ve a choice as to when 
eagji section will be done. 



"PTQ^'(2) ^ ' ^ , • 

* ^ This is a veryLshort Ife^on on transfer cases, sprag\inits, » 
and power take-off units. Each subject area is contained 
in a separate module which can 'be taken at any time. The 
* • "end of lesson test can be attempted only after all three 
sections have been completed. 

fhe remaining lessons are virtually identical to the earlier versions 

as far as their organlzat ion-and structute are concerned. *In other 

seqtions within this review, specific commepts will be made where the 

lessons differ from their, predec^sors. , 
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Specific Conments 



Teaching to an "Objective 



^ Vl n-,/ *"! 1 cases the objective has reTnained the same:' to pass the epd of 

lesson test with more than 75-80Z accuracy. In "starter" (2> , thlt^^bjec-' 

tlve Is stated, although It was In "starter"(l) . The accomplishing 

of the objective Is based solely on passing a^end^of lesson test. As In 

♦ . 

the Overview, -much attentlan%%rLll be given to the end of lesson test 
and to t>he factors affecting the student's successful completion of It. 
Terminology 

The Overview provided an extensive analys.ls of this subject^ ^ 
As far as the current report Is concerned, "transmissioh"(2) Is the only 
lesson to show app^reclable changes in this category. (See Appe.ndix VI 
for details.) Usually the^coirt^xt In which the key terms/ concepts appear 

must still be used^fpr explanatory purposes. There has been no slgnlfl-- 

^ * * . / 

cant shift- in emphasis toward rigorous definition of terminology in the 

\ ' ^ ^ 

lat^r series. > 

Interaction 

In general the amount and type of Interaction Is very' similar In 
both sets of lessons. However a few questions have-'been added or rephrased 
«a4' will be mentioned because'they relate to the Items on the end of 
lesson, test Frequently the phrasing change or addition brings the 
qu^rion more^closely In line -with test Items seen later. Comprehension 




1 r . 

However, the criterion for a lesson that was not part of» the group 
of eight examined In this report, Engine Lubrication System, .was relaxed 
during revision- from 7 of 8 (88%) to 6 of 8 (75%). 
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of the questions may also be improved by this remodeling, ^^xamples o 

these are a*s follows: 

. '*emission"(2j > ^ 

One multiple choice 4uestion has been added; it is on 
"^^"topic relaLed to* the subject matter stressed in a 
previous' question — air flow around the draft tube in a 
non-positive crankcase ventilating system, 

The author rephrased a completed question concerning 
the location of the entrance point, folf air in the 
positive-open type of crankcase. ventilating system. 
The old question^ which the rephrase replaces^ wa^ very' 
similar to a previous question. Thus ^he rephrased 
^ question is irapprtant because it eliminates a nearly 
«redundaivt .^uas^^^JiT^and instead emahasiz^es previously 
unstressed knowledge. ^ 

"ttansmission^' (Z) 

t 

Four questions have been added: three of these cover 
\ topics 'for which there were previously no questions in 
the body of ,the lesson,' but which were covered in* the 
te3t. T^e other addition is less significant because the , 
material it stresses is al'ready covered by another question,- 

. "dTlveshaft"(Z) 

• , A single questiorl has been reworded so that it now relates 
to material in one af ^the, test items. ^ 

"electricity*' (2)' " - 

! , - ' ' * 

' One question, has been changed from an open-ended fill-in 
("What would you call the smallest piec^that would still 
be the saipe Kind of matter? — an,y good guess will do)" to 
V ^ a-^hetoriqal question ("What is a molecule?"). .A discus- 
sion of that subject area then follows. 

In general the questlj,ons added or remod^eled are dijectly related to 

items On the test and apparetitly stress important concepts. 

Visual Presentation 

Ii!t the majority of cases thj^raphic displays and other ^sp.ects 

of Che'visual presentation have remained the ^^f^rte in the second set o 



lessons. However a few changes can |^ detected: 




^ "emissionU2) 

One small change is evident in Chat two 3niall phraSes 
are underlined which stress an important component of 
r one of the crankcase ventilating sys^pfims that is 
reviewed on the end of lesson test. ^ ' 

^^starter"(2) * / ' - - ^ 

Frequently the text descr'^ing a slide is superin^osed 
, over the colored portion of the fiche-, making ^it diffi- ^ 
cult to read the text. This, is the case, whether the 
old or new fiche ±9 used. Also^in some cases it ia^ very 
H difficult" to discern what the slide .is presenting: 
there is no lettering on the fiche indicating what is . 
being seen. Upon reading' the above comments in June/ 
July 1976, th^^hanute staff replied that they felt all - 
such problems had been correct-ed by reprogrammipg anji 
' . by production of 'a new fiche. SHowever, the reviewer ^re- 
checked the lessons and f-ound many of the problems 
still in existence. * _ , * * . 

"diesftlV(2) • • ' ' : 

Replacement of a mediocre and .tedious graphic at the 
beginning (i.e., something like a cross between a 
computer: ana a stock repcbrt' ticker tape, perhaps an 
attempt at burner?) with a slid^ of a diesel engine. 
Bven the author realised the ,i,mpact of the original 
since he allowed the NEXT* ke^ to bxeak through the 
, . plQj:ting. . 

^ ' . ^ i 

"6lectricity"(2)^ . - , 

p Important terms/concepts in several units of text are 
''stressed by the use of all capital letters. 

Thus these c^ianges appear to be perceptible Attempts at heightening 

visual impact of the CBE presentation. » . . , ' 

Feedback . . . ^ ^ 

.' * ' • 

J ■ 

/ Some changes in feedkack can be noted here: 

"eml88lon'l(2) " ' • • . ' • 

» In one unit the nfev/ Material that once appeared in the' fee 
back position, althougT>not properly feedback, now' appears 
i separate unit/frame. (More details on this placement'^ 
new material in section; on , "feedback" in the Ovarvlew . K 
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"starter"(2) ♦ • * ' , " 

' Throughout this lesson the answer is given 'in the feejd- ^ 
back after three incorrect tries. This enables tlife ^ 
^^^^^^ student tp move on without halting his progress when ^ 
he 'is unable to proceed (ajthougli this technique may 
encourage 'a careless a*ttitude in the student and thus 
has some hazards). ^ - - ' , . 

"transmission"(2)- ' * - 

In one'unit new Tnateri^l is^ taken from thQ feedback 
^ ' position, th6ugh it tod really wa^sn't feedback, and put 

into a: separate unit/frame/ Lf new'material is presented)' 
to the student at the time he i-s given or is expecting 
^ feedback, he may pass over it l^^ghtly and thus may miss . , 

valuable information. This seems especially likely to , - ^ ^ 

happen, if th6 student h^s ^S^^ exposed to" intimidating 

or abusive feedback tn dthe^^«RRons. " ' ' " ■ 

Tests ^ ' . ^ - 

The end of- lesson tests are' the sole criteria for accomplishing ' , 

' the; objfCtiVe in these ^lessons. There' has beea considerable remodeling 

-of these tests,' as seen in the comments below: ' ^ ' • 

"emis5ion"jC2) * ' ^ , - 

*rwo questions on the test have been rephrased, foip' clarity. 
See^ Appendix III for mor^'^Titf^rmation on this subject. 

\ "starter" y) ^ ' 

The ortLginal end of lesson t«st in "starter"(l) Ijas been \ i 
made dhe end of lesson review ccw^taining seven multiple | * 
choice! questions. The new test is- the familiar ten- * | 
' item nktching drill. From a subject-matter point of viev^, 

the orjiginal test seeme^ more difficult aVid jnay have also] 
been ^ mote valid test of the objectives.' ^ ^ 

' ^'transmis 4on"(2) * . ' ' ' ^ ' ? ' 

The sc^jing proc^ure should be mentiolned since it | , 

diff^r^a froih that in tha previous version, Tn "tran.s- " ^ 
mission" (1) the corre.ct and incorrect answers were shown 
after/,the test wag graded/ Npw there is no individual J.' 
sfcoDli|g., but the -student is given refresher information ^ 
oh Questions he* missed: brief •'payphrases of previously ^ 
% seen, mater ia'i or the material itself is shown again, \ 
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, Then- the student is taken back to the index page with 
- ^ specific sections^ marked that muat^ be completed before 
*^ h% can retajce the whole test. This is another variation 

on the forced feview technique, which j^y be effective 
^ as a pedagogical device. ^ ^ 

' "dieserH'2) ^ ' . ' . 

' $ * ' ^ / 

At several places in . the tpxt there are stpong hints 
that .certain items should be rememberedl^^ Hence though ^ 
the test itself has not changed, its diff;i.culty has 
. ' possibly been'^educed^ 'Fox more on this, see Appendix II 

to this 'report. ^ 

. V • ' ' ^ ^ 

"drive shaft'X2) • - . ^ ^ 

Only two of the ten questions on the end of lesson te^t 
are the same as in "drive shaft"(l). The interaction 
in the lesson now follows more closely the information 
requested on the eqd of lesson test, although the ^content 
• , is roughly the same. * . ^ ^ ' 

The following table is a survey of the changes made in the 'reldtion- 

ship of interaction and items on the test between the two series of 

lessoi)s. ^The percentag| indicate^s^^^e number of tes^t items that have been 

previously covered via interaction in the body of the iesson. ^r example, 

in the case of "^tarter" (2) , there is at least* one paraphrase question in ' 

tl)e tfext for. every^uestion on th§' endwof/ iessoo t^st. It was not 

considered necessary* to make ^ s'eparate listing for tHnse instances where 

a question on the end of lesson test was anticipated by more than one 

i . ^ ^ ^ j • ^• 

question in the body of the lesson. The Letieflt was thought to be 

problematfcal, ' 

" Related observatlor^p In the reviewjer^s opinion, the reading 

ability, dif'ficul'ty of questions, length bf leaf5on, etc. rjiay have an 

important effect on the performance*of thje students: e.g., "driveshaf t" (2) 

is very brief (about 20 frames). Thus even if the percentage is not 

high, the student may do better than expected if the same material 

6S • ^ 
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Percentage .of*^ Test Items Covered via. Int^Ya'ction 
Lesrson 



Verglon'(l) . Change 



1%- 



. , emission 
Starter 
transmission' 
dies^l 
««hydraulics 

s 

3 ^ 4rive shaft 
electricity" * 



40%" I'. 

20% 

40%' 

30%" • 
100% 
/•40% 



+30% 
+80% 
+50% 
+20% 
• 0% 



+30% 



Version^)' 

70% 
. ^00%' 
90%' ^ 
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|5% 



100% 

e 

70% 



40%' 



,85% 




■ — ^— — ^— ^ — ^ — 

This .iacltideS one .questiou^on the t'est^not c*overed 
at ^all via interaction &r text* ExaminatitJn of the- test 
•{esul^t'S shows* that the performance ot the students on this 
q^es«tion^ Was* about the same -as tlielr perfoj(lipce on.-otheii^ 
dfst questions I ) * % 



ded a longer, lesson so thAt short-term rfetentlofi would not be 



c^r in performance on the test. . . . ^ 

' It should be mentioned tHat this Is , not presented as any definiti ve 

^ t 

judgement '*bn' lesson quality.- It is^l'ikely that . the questicyis^n the - 

end of lesson tests wtfre modified in an "attempt to m^ke them easier so . 

that stud^Ttifes *=tould pass the tests and the -lemons ^duld vafidat^. Also' 

the reviewer Kas no way of determining what was cpYisidere'd i)riot knowledgev 

: , • ' ; ^ 

for th^ students* in going through the lesson. ^Thus someJlerms ^nd/or^ 
conc^p.Cs may familiar to the class and may hot have, been stressed due 



to this prior exposure. Consequently the students cdi^ld' aYiswer the test 



qi^stipns even wittiput their being pre-te*ste#^ -In. this >?ay/ ^acfors 
outside tl\e boundari^ 'Gf ^the lessLon* may play an important role in the* 
success ^TflR^l'esson 'as d pedagogical whole. 
Coding 



Major 'problems > ' * r ' ' 

, ^'drive 6haft"(2) ^ " - « * . 

s ' On .the index ftage, if a student presses any key other thati^' 

• * - the ones allowed (i.e., a, b, c; cf, e,) ^n execution error \ , 

' • occurs: . the 'student ' s progress through the lesson i»s' • - 

interrupted, and he is summarily- deleteii. This' error .wa^ 

NOT pregent in "drive shaft"(l),»' ' 

^ Minor problems .. Some minor prbblems were <Jfscovered in the^ caurse of 

• V V- * " 

preparing the individual reviews. In general, it is the policy in this 
report to list only changes tha^ ^ave been made from the j)i^vious series, 
Ho>?ever irt this section on coding, uncorrecteo^rrors ,are signif icantX^^ 
*since thej^. may affect^ student . progre&s through the lesson. Thu5 they will 
be noted here when they have beAl passed Ctfi iinchanged- from the eardy ^ 
version. A mote extensile descFipt^Pn of coding proW.ems in general in the 
Chaaute lessons is a.vallable "in the Overview. 



•. , * V . 

Specific examples. 

. "emission" (2) . ^ ^ . * ' * ^ 

: . ^ \ . #^ 

^'There is only part ial"" erasure of a questiop when forced 
review is done for an ihcorrect response. This -is [ 
, uncorrected from previous version^ 

^wb '*Pr^ss NEXTs" appear^iin the, unit ip. which the percen- 
tage of correct answers on te*f' appear^.* This is uncor- 
rected from previous version., . 

. Pm omission of a^xritical commaVic^^ iti certain corTte)its 
may lead to-^n 'inadvertent returff'to the beginning of 
the lesson and logs of about 15-20 minutes work. This 
is uncprrfected frojn previous version. 



"•starter" (2) . 

Because of some faulty coding,^ part of the section on a 
"Contr5l Circuits" is not functional. I&ue^^ to the r 
unfinished condition of ''starter"(l) , this problem 
^ could not be detected in the early version. 

transmission 

Spme minor non-era*s\ire and overprinting .occurs, ^ especiall'y 
• in the feedba^ position. This is uncorrected trom- 
^reviQus versi^- , ' * . ^ 

"diesel"(2) , ^ ^ 

Some minor overprinting of^ lines of text. occurs. In one 
case a slide is available via LAB but its. availability 
is nbt indicated for the student. Thfg is uncorrected 
from previous version. ^ , ^ . " * 

' ' ' \ ^ 
**hydraulics"(2) ' ^ . . * ' , 

The f4>rced review for ©isging a d^-ili^^irem now worlcs: 
it was'optional in "hydraulics" ( D^ue 'to ft minor, coding 
\ i problem. In the same drill wherf.the studejit misses an* 
item .three ti-mes, the correct answer appears, butj^.it is 
ptin^d over a subsequent item 'in, the drill* Thus it , 
Is difficult to' read the. correct a'nswer.. Thi^ is 
uncorrected from previous version. 

"drive 3haft"(2) 

At various places in t4ie lesson, C^^e is a long wait for 
the student while special character^ are loaded: this, 
impedes the progress of the stpdent. * ^^re are total of 
jjsix possible loadings of special chfraoters. which must 

occur befor^iAbe whole lesson is completed. Ht^merou^ (?) 
^ coramerfts -(i.e., "PLATO has now prepar^ some strokes,")/ 
attempt to fill' in^ the delay. This is the ^same as in 
' T)reviou^ version.* '* * ' * ^ 



"PTQ"(2) 



bue to the unfinished condition of "Wo'^'d), all of the 
^llowing are new problems. . * 

"A fr^me seemingly designed as ^n introduction (as deter- 
mined both by its content anH its^name — "intro'*') is in ^ 
fact Imvisible to alT st'j^ents exce^^t thpse who fail a 
particular question* Then they are*feenO back tQ th^s' unit 
as part of a forced review.* \ r * 



. One "jqll^s^ion reques'bs the stAident to name twc^ kinds of ) . 

transfelf c§ses but the judging "^tquence will count ^ithd^r ^ ^ 
^half of the answer as completely correct. A very minor . 

I change,' JIndicdted in the pi^int-out ,^ill prevent from • 

occurring. ^ ^ . * ' " 

' ' * ' ' ' / • ' 

A "'PRESS NEXT'** appears at the bot*tom of the page but a 
s^'tudent response jauat be made^ before t\\e student can * - 
mofe on £arom this plac-ie. \In essence , ' t;he NEXT* key is ' ^ 
non-functional at *this point., and .the directions can< , • 
cofcfuse the student. . 

■ ^\ '• 'J^- . ' • . 

- r •• ■ . ""^ . • • . 



"electricity" (2) , - , - s' ' „ 

' • " ' , ' . ■ 

"Pressing. LAB at one point takes the st-udent bacJc' £o the 
beginninApf .the t4st. At that pointy. the directions in ^ 
"^lacge letters, "DO NOT P^RESS NEXT'* , 'overprint on '"Press - ^ 
i-NEXT-"- in* normal size writing. -^ThTg is a new problem^ 

Summajy Otl^ode ^ ^ - ^ ' . ^ ' 

* . ' * ' ' ' ' I ' . ■ ^ * . 

^ Thus it \is evidaA th^t some .problems ,* most not of a .serious' natute 

' ' ' 0 ' , f 

remain'in thes£; Tessons: Tttese errors mfy prp\^ to be a *nuisance ''or a ^ 



source of* confus\on tx) students wprkiag through the lesson. Of the minor * ' 
pro])lem^, none would require extensive re>?orking of the code to cgrrect.r /\ 
Indeed some problems, such ^ the overprinting in "electricity" (2) above,, 
had been corrected as of September 1975, when Varian copies of ^he block ^ 
directory pages were made fpr these lessons. 



4 



As part--of the ov&rall Evaluation of^Chamite lessons, the MTC staff 
has monitored C^e .use of Wese Lessons at .Chanute and at; Parkl^d College. 
We were somewhat surprised to 'find that execution^ errors were still 

* ' r . 

occ4i.rr^ng /af^r^lessons had been used by several hundred ^^^j^de^s. ^ Undei^ 
ordiaaify circumstances,. 20-50 students should discover all errors. «' 



Although' a number of the 4}roblems napoq^ed by ^f^C staff have been fixed, 
some "" — — — - — ^j^^ _ 



me ^*'haye reitf^rjed and continue to -cause execution eyrcrrs for other 
Students. ^ Minbr problems of ^verprint ing can db ^xcus^ed by dnsuf f ic iept 



67 



1 



maapjpwer to ^'polish'* the lessons% biit J.t *ls. inore. dif f icult to/e'xplain , ^ 

n ^ * ' . ' , • 

away more setious problems that interrupt progress iir the lesson. 

New material » ... " * , ^ 

' 3A. apall amount of .new material h^^ been added to Jthese iessor^. In 

mbst cqses it^consists of ''teaching points,*",' reworkings of old questions, ^, 

and rephrasing of some Items, of feedback; These have already been 

. ^ •/ ; ■ ; • - . 

discussed Ttn- the Overview (sections on General Comments: Components, , 

Specific Comments:*, Interact^ion and Feedback.)^ In some cases, whole uRits 

of new infdrmafeion have 'been "added: ' ' • * * 

^ • ■ • » 
"transmission" (2) «, ' • " ' ' ^ 

A small un^'t containing ijew material on the .torque cortverter . 
^ and fljuid cpupling' hap been inserted- here. 



t 

li- t c 
teri 



Du'^to the ujifinished s^at^ of "PTO*'(U, almp;?t all the 
material here i6 new. " ■ • * 



I 



.'7: 



"l 
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Symmary and, Cojicluslons * / 

^he various categeries dl specific conmtents'can be st^BlFized as 

* • * * • 

follows: . - * ■ 

• 1)- The by^pass APt ion has* 6een 'el iminated in^all h\it one, 
'•• Qf 'these lessons — '^transmission" (2) • 

•s. 2) The obj^ecfives of these less'ons have not changed! 

' • V^) The amount of interaction specifically reJtfeted to the 

, " end of lesson tebts has increased- \ - v 

■ ■• ' s I 

,4) With certain notable, exceptions, codtn^^roblems have 

ceased to be an important factor in th^^se later lessons. 

,^5.) In several ca^Fff^^w material^ has .been removed tvom 

the feedbacj^^osit ion and put into' new separate unils. 

6) All lessons /a re fully operational and have their full 
m ^complement of complete unit^ •- ^ . 

' . \ . • * ' - . " 

Based ' on t|^e way that tti^se eight lessens chaoged^f r(|n their pre- 

. • • > ^ ' \ 

validation to their val^idated form, one c^n characteri;^^^he ^process of 
^ ^ ^ • ^ / 

revision to improve validation capability as follows: 



1) Lessons which validated quickly were not appreciably 
changed; typically 'only cqding e-rt^ffs'were c:o^?ected.' 

2) <vOng lessons which'did not* validate quickly were dividejd 
intO'2-4 pieces. (This informatipn is based in part ofi 
conversations -with Chanute staff auttfor's and data not 
found in the %tglit^ lessoas examined in' this report ;) 

. N / . t> •■ ^ • ; . 

3; In g^tieral. the crbjeCti^ej or criteria' were niDt changed.. 

4) By looking af item results ijj^om the end^of 1 es son^te^ ts ,* 
questions with /:oasist:ent ly low performance wA^e'iden-' # 
^ . tified» In some cas^s, these itenrg'or the whole tests 
^\ - ' were rewritten in most cises, infornifitioo to aid ^tu- 
I ^ ^ dent performance those iMms v^irs stressed in^^he body 

* of the lesson. Emphasis was provided by giving the s-^u-" 
' ' ^ d^nt questions siitiilar io 'those 6n^^th« enci of lesson test 
• ^ When an item was already covered wi^th a, question in^ the 

'* , body of the lesson, the feedback warneS* the student to 

remember 'the answer for the test. » 
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5) When the student was having difficulty answering the 
que^ion in the body of the lesson, correct answers and 
important terms were accented with various special 
•display techniques. 

. * 

6) Indices were provided .ta allow the sti^ent self reviews', 
^nd some lessous *were reorganized, possibly to improve 
the flpw of the lesson, . 



7) 



8) 



Minor problems — such as^ overprint ing^^^sevefal layers 
of text, partial erasures, or ucn-fuMHbnal instruc- 
' tions to phe student — are not fixed unless they signi- ' 
ficai>tly interfered with the student's pro'feress. 

Execution errors still occur afjter more than one year 
'of studjent use. 



\ 



7 



Iff 



\ 
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Qbservatio^ from Chan|^te 



1) V Many chfeges were imposed to attempt standardization 
of lessons (written prior to June 1974) • Durin'g IS.D 
development, most effort was expended on d(^w lessons;* 
» V ^"'^H^. ' the prepared lessons were not modified until v^lida-. 



tion. 



4 - ■ 



2) * Many changes were, the rfesult of the init*ial use of poor 
. programming techniques. ^ 

3) ' From the Chanute evaluator's observations of the effort 

and others, it seems implicit to the instructional pro- 
gramming philosophy cKat students will lear^l in spite 
of the techniques used. Consequently the importance of 
validation techniques — when considered as a group — lay 

^ in the .-debugging of a lesson, making it usable for the 
majority of students- Though it seems like heresy to ^ 
observe cases of 
^ '< * • 

^ a,* reduced standards, 

b. * cuing students to test items, 

c. ' dividing lessons to facilitate validation, 
^ , d. pretest Item practice, and 

e. reduction in item difficulty, ^ 

■ ' r ■ • . ■ ■ r 

a perjorative impact on lesson performance outcomes 
was noi, noted. Validation techniques are arbitrary in 
many cases, aad there^^re can be flexible, for the end 
result is a viabfe product . 4 » 



4 



(• . The eyaluator Is no|r speaking, of .the fend of lesson test-*^ob\iously 
xhese should improve — rather he refers' to the block exam^i instrucVof 
* assessment , ,field evaluation, etc. ' * \ 



r. 
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• ARpendix 1:^' The Eight Chanute* Lessons Selected 



/ 



emission(l) 
starter(l) 
transmission(l) \ 
diesel(l) 
hydraulics(l) * 
drive shaft (1) 
power take off PTO « mtcc7 
electricity(l) = mtccS 




emission(2) = mtcclO 
starter(2) - := nitccll 
tranaBission(2) « intccl2 
diesel(2) » intccl3 

hydraulic8(2) = mtcclA 
drive 8^ft(2) = mtcclS 
PT0(2) » mtccie 

electricity = mfcclS 



ERLC 
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Appendix II: Stressing of Enj of Lesson Test Items 
via Int eraction in "diesel"(2) 

c . ^ 

The following table give« information the relationship \^f 

interaction to test items ia one particular lesson. This lesson was 

chosen because of the, particular way in whiah test items are stressed- 

* • 

in tlj^e;, bo^dy of th^"^esson. Each '*X'' represents one occurrence of the 

designated characteristic. The stressing of Information can be divided 

into specific categories: 

Z percentage of students^ answering question correctly 

' ^(79 cases total) 

i/a(dir) = * virtually identi«cal questions in the body of the lesson 

i/a(indir) = related question/answer stressed by interaction 

text 



f/b 



information for answer provided in text, but pot 
emphasized - • * 

answer to this question provided in the feedback to 
another questicnn 



blatant 1 = feedback to a question) gives "Remember this for test." 

• It appears in "emissiah"(2) but NOT in "emission" ( 1) 

blatant 2 « strong waraing in text: "Don't forget . (this)" and/^r 

a'nswer set off from normal text via unu8iy|i positioning 
or special characters. It also appears in the later 
version of "emission" only. 



1/ o 
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% . 


i/aCdir) 


(i/a(indir) 


text 


f/b blatant 1 


1 


• 109 


■ 


* 


X 


• 


2 


85 


X 




X 


. X 


3 


99 






XX 




4 


94 




X 


X 




5 


75 






X 








t 








6 


. 71 


« 


m 






7 


90 






XX 




8 


89 






X. 




9 


'85 








X 


10 


*85 , 






X 





X 
X 



Summary of and conclusions to be drawn frotn Appendix li: 

1) . Exira assistance is provided »in the form of strong 

hints to remember certain bits of data. 

2) Some of the answers to test items appear at oblique ^ 
places in the lesson (i.e. , within textual passages or 
in related questions)., ' . 

3) As mentioned above, some^are hinted at so strongly 

^ it is as if th^y were stressed before the studea^ sees 

them on. the fin^l tes.t. •** '"^ 

. . . 

4) . Two o'f the questions (//5 and //6) are given very hedvy 

stress by the use-e^vstrong Hints: thus .their j>ercentage , 
correct figurejs ina^'^h.ave been inflated. 
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Appendix III: Modifications of T&st Items irt "emission" (2) 

There are two tesrts: a ^inal quiz (no longer equal to a by-pass '* 

> 

test) and an end of lesson teist. The former is unchange'd; in the |pd of 
lesson tes|^ however, there is a slight rephrasing of two questions whijch 
makes for clarity akd J.bss collusion for the student looking for the ' 
right answer* Comparisorr'^-o^^^th^--few^ may help to show what has been dpne 
to improve the precision of the questions asked. First the questions from 
"emission" (1) : ^ . . > 

' . / * •I " . 

2. Component that uses PCV 

engine vacf-uum to dra-w ♦ ^ 

* gasses from crank^se 
' intake manifold.^^ 

31*. Vane-type ratary a. air pump' 



J. 



t 



In //2, the phrasing is misleading: it is actually ,tbe vacuum that operates 
the pcv valve; the valve does not "use" the vacuum, l^n* the -f oyfbwin^ 
question, there is a problem: another possible maA^^, "K." air injeq||lon," 
would be just as likely a choice, and as correct, as the one allowed, 
"a, air-pump." • " .^^ 

^ "Ehe followin^^ are ^the .corrected versions-'of the questions as we-l 1 
as the changft^ in^ the phrasing oT thqr^correct answers 



2. ' , Valve that is operated d. PCV valve 

by engine vaculim and ^ 
draws gasses from the 
*crankcase to -the intake 
manif ol^. ^ . 



feet answers: . ' • ^ ^^^^^^'^y^ 



Type of pump used on air 'a. Vane type 

injections system, " rotary* pump ' 



ERIC 



' i 1' 
♦ 



The author ^eems to.be aw»re of the original vagueness in the .questions 
in *'emissi(gp*'(l) and has taken steps to corr«a:t the probletn. Also 
the appearance of "valve" and "pump" in'bo^h the description and the . 
desired answer is 'a strong prompt. , ^ 

Results . The rephrasing of the two questions and the addition of 
a question, as mentioned fix the section bn interaction, increases the 
total ^f test items previously stressed Jlpy interaction to 70% in 
"emission^'CZ). . * • ^ . ^ ^ 



* 

V 
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- Appendix IV: Teaching Poi/lts • «. ■ ▼ 

The following is at table listing the occurrences or absence of a- 
section called "teaching points'* that appears in some^-of these |fessons. 
If ppes'eht, the section is only accessible to ah author, NOT a student, 
via^ATA*"at the appropriate place in the lesson. In some cases, 

dire\£tions are given as^o,its presence. In others it must be discerned* 

r . ' * • ; 

from an examination of the printout. » ^ ' . " 




< 




* 




A. " 
Lesso^' 


Present? 


t 




emission (I) 






emission (2) 


yes 


starter ( 1) 


no 


starter (2-) j 


yes^ 


transmission(l) 


no 




m 


transmission (2) 


yes 


dl^sel(l) ^ 


no 


djesei^2) 




hydraulics(l) ^ 


no 


hydraTll'i<!:s(2) 


yes 


"drive shaft(l) 




drive shaft (2) 


, yes 


PTO(l) 


no 


PT0(2) - 


yes 


electTicity(l) 


yes 


electricity.(2) 


yes' 











Directions given? 



yes 

no 

no 

"yes 

no 

no 

no 
no 
no 
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Afiifendlx -V:. Lessbn^ Revj^ewed f or. thjs Report Available 
/ In a^"fj.xedc' a^<i "unfixed*^ Fona . • 



' r % 



"Fixed'Varid "unf i:ced** Lesson Names. 



. ' , Lessob 

e^ssfon 
V ' ^ starter ^ 

! "Cv 3, 

' ' ' • transmission 



Fixed' f omr 



4 dt^sei 



I 



' * = " * Unflxled ^oTin 

',^ha3 some ^rnkt-e rial ^rom ch'a63 ' 



Imtccll : . 
mtccl2 
mtccl3 . 



cha4^ 



cha73^ - ' • 

; ^a74 ^^end lof les^op test in chal3 + 



^ 1 



, * ;^y#auliqs 

*dtlve sjiaft^ 



power take off i 
* electricity ^ ' 



, 'mtccJ4 V 



' cha78 4 sojne material *in cha80/§A 



- ^rcc 



:cl8/ '^ cha9Z + other lessqifs ll^edj^ in indc^x \ 
' v"**' \ as sections: . <:>ia37V^ IQjCI,' and^ 43 - ^ 



. Lessons -in which both '^onfas are b,^ Ch^*s^me a^ljhor. 
^T^e ahswfers to '^hje qaelstion^ ; wfiic^puwe strongly hinted atv (sefe- > 



' /* . ^^j,-App^ifdix- II) ar^ foynd. in th&i "cha" l^^ss^jn; "thus^jdofnman^'as to when tfeey - 
^.^ ^' - were /inserted and" boir weir sfudejits. performed <)n -them x(l5tored liWata ; ' * ' 
' ' '.retords) ^re d^fcf icui t' tp /correlatelSue to thje! "jidh-f ixe^**- 4ta te.^f ' <ij|€ ' ^ 
lesson! ^.^ r-^'^.* • • ^ . 



'IB. 



' j^-; ' * . - ^> , Sopie', adiditfonairKoimnents' Co kee1p'.it\^'f»itid,<*'> " • ' ^ ' ^ / 

. - IJ,^ the- sjequence moye^ni^' of the* ptodep^" iS" ^I'ora' t>he ' . " 
. \, 1^ first '*c>ia*' lesson' to'^H^ oth^t^ ^a/ a'. ;;^jum|^ ^ , 

^•ssoTjs, .have .beed^^bjeqrt€(| "to f/equent' ^ 
' . •■ .• y.\,-.. * edtti,ng so tJ\at/ th'ey »^te HGT.-j^ix^ct 'etit^tVal^nts^^. . ' ' ^ ■*/ 
¥i " ' t'K^ir rtltc -count eVDraftfe.* , " ' . ' ' '™ " 

ERIC '^- k. * * P-^-^. . . ' 7. 



78 



Alppen-dix VI: A Siirvey of Major and Minor Changes' In tH^se tAsoi?» 




• 



♦ 



4 



f 

wesson — 
■ ■ ■ ^ C- 


N 

c 

^ GO 

■' ' >^ 


60 

•e 

u 

* 


CD 








c ^ 

"0 , 
0 , 

o 


^ c*mission-/ 






X ■ 






& . ■* 




^ starter 


* X . 




X 


$s 




X 

\ ■ 




ll^ttansmission 


X 


X 


• .X- ' 


X 




X 




— * ^ 

diesel 





f." ■ • 












iiydr^ulics ♦ 




— t 


•1 

\ 








' X 


drive shaft 








— i — ^ — 






« 


PTO . 


! X 


'x 

<» ■ 


X 




t X 

f 


X • 


— 1» 

> 


©lect^rigity 








X 






1 - 



0 



E]q5lanation : x • changes of major ^import anc^e (one more) 
T^jchapges of minor importance ^ ^ 



^j^^ jcha 



Several changes were nor noted becaixse^hey seemed' o]5viously to be 

■ ' - - >:■•,.'• 

the completion of previously unfinJLsbed work rather than modif Icati^hs 
based on student usage. For example, the two le^sojns — *'starter**'(2) . 

^o^ 



**PT0"(2) now have objectives; tne e^ly^ verd'tons'ffd not. <uo^ included 
,^re two "acrcxss the board** changes affecting essent ially ^every lesson: 
removal of by-pCas's test,^, except for *'dieseV*(2) , and a<}dttion of teaching 
points accessible to* instructore. . , ^ 
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J. 
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Conclusions : 



*1>, "?T0" was .changed' considerably whil^ "Hydraulics" was 
relatively unmodified. 

2) "Startler" and- "t transmission" reflected extensive changes 
' whil| *'diesel"^ and *^drive. shaft" did not. 
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